Systems and methods for detecting and assessing distracted drivers

ABSTRACT

Embodiments of the present invention meet this need and others by providing systems and methods for detecting and assessing distracted drivers. Embodiments collect vehicle and driving data using a mobile device of a user. In a particular embodiment, data collected using the mobile device is analyzed to determine when the user is engaging in distracted driving behavior.

CROSS-REFERENCES TO RELATED APPLICATIONS

This application claims priority to U.S. Provisional Patent ApplicationNo. 62/219,989, filed on Sep. 17, 2015, entitled “METHODS AND SYSTEMSFOR DETECTING AND ASSESSING DISTRACTED DRIVERS”, U.S. Provisional PatentApplication No. 62/318,983, filed on Apr. 6, 2016, entitled “SYSTEMS ANDMETHODS FOR DETECTING DEVICE USAGE”, and U.S. Provisional PatentApplication No. 62/353,455, filed on Jun. 22, 2016, entitled “SYSTEMSAND METHODS FOR DETECTING DEVICE USAGE”, the disclosures of which arehereby incorporated by reference in their entireties for all purposes.

BACKGROUND OF THE INVENTION

Mobile devices, including smart phones, have been utilized to providelocation information to users. Mobile devices can use a number ofdifferent techniques to produce location data. One example is the use ofGlobal Positioning System (GPS) chipsets, which are now widelyavailable, to produce location information for a mobile device.

Some systems have been developed to track driving behaviors includingspeed, braking, and turn speed. For example, external devices that pluginto an automobile's computer system have been integrated with vehiclesto track driving behavior. One driving behavior of interest is adriver's inattention to tasks that are important to driving. One sourceof driver inattention is the use of a mobile device while driving.

SUMMARY OF THE INVENTION

Despite the progress made in relation to collecting data related todrivers and their driving behavior, there is a need in the art forimproved systems and methods related to tracking and evaluating theinattention of drivers.

Embodiments of the present invention meet this need and others byproviding systems and methods for detecting and assessing distracteddrivers. Embodiments collect vehicle and driving data using a mobiledevice of a user. In a particular embodiment, data collected using themobile device is analyzed to determine when the user is engaging indistracted driving behavior.

According to some embodiments of the present invention, a system isprovided. The system includes a mobile device comprising a plurality ofsensors, a memory, and a processor coupled to the memory. The processoris configured to perform operations including obtaining a plurality ofmovement measurements over a plurality of time intervals from at leastone of the plurality of sensors in the mobile device of a user using atrip in a vehicle. The operations further include classifying a subsetof the plurality of movement measurements as mobile device usagemovements based on characteristics of the subset. The mobile deviceusage movements are indicative of usage of the mobile device by theuser. The operations further include filtering the mobile device usagemovements, and outputting the filtered mobile device usage movements.

According to some embodiments of the present invention, a method isprovided. The method comprises obtaining a plurality of movementmeasurements over a plurality of time intervals from at least one of aplurality of sensors in a mobile device of a user during a trip in avehicle. The method further comprises classifying a subset of theplurality of movement measurements as mobile device usage movementsbased on characteristics of the subset. The mobile device usagemovements are indicative of usage of the mobile device by the user. Themethod further comprises filtering the mobile device usage movements,and outputting the filtered mobile device usage movements.

These and other embodiments of the invention along with many of theiradvantages and features are described in more detail in conjunction withthe text below and attached figures.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a system diagram illustrating a mobile device of a mobiledevice usage detection system according to an embodiment of the presentinvention.

FIG. 2 is a system diagram illustrating a server of a mobile deviceusage detection system according to an embodiment of the presentinvention.

FIG. 3 is a flowchart illustrating the capture, analysis and use ofattentiveness data associated with a driver according to an embodimentof the present invention.

FIG. 4A is a flowchart illustrating a mobile device usage detectionmethod according to an embodiment of the present invention.

FIG. 4B is a flowchart illustrating a driver identification method usingmobile device usage movements according to an embodiment of the presentinvention.

FIG. 4C is a flowchart illustrating a driver identification method usinggravitational acceleration movements according to an embodiment of thepresent invention.

FIG. 5 is a flowchart illustrating a mobile device usage detectionmethod using usage probabilities according to an embodiment of thepresent invention.

FIG. 6 is a flowchart illustrating the inputs into a global positioningsystem (GPS) active detection engine according to an embodiment of thepresent invention.

FIGS. 7A-B are graphs of actual mobile device usage intervals andaccelerometer measurements during those intervals when a user is usingan application and taking a phone call according to an embodiment of thepresent invention.

FIGS. 8A-B are graphs of actual mobile device usage intervals andaccelerometer measurements during those intervals when a user is usingtwo different applications according to an embodiment of the presentinvention.

FIGS. 9A-B are graphs of actual mobile device usage intervals andaccelerometer measurements during those intervals when a user is usingapplications and text messaging according to an embodiment of thepresent invention.

FIGS. 10A-D are plots of sensor measurements and predicted usageintervals according to an embodiment of the present invention.

FIG. 11A is a plot of predicted mobile device usage according to anembodiment of the present invention.

FIG. 11B is a plot of actual device usage according to an embodiment ofthe present invention.

FIG. 12 is a diagram of a mobile device in a vehicle according to anembodiment of the present invention.

FIG. 13 is a diagram of a driver and a mobile device in a vehicleaccording to an embodiment of the present invention.

FIG. 14 is a diagram of a mobile device in a cup holder in a vehicleaccording to an embodiment of the present invention.

FIG. 15 is a diagram of a driver, a mobile device and accelerometer dataaccording to an embodiment of the present invention.

FIG. 16 is a diagram of a driver of a mobile device held up toward thedriver for screen viewing according to an embodiment of the presentinvention.

FIG. 17 is a diagram of a driver and a mobile device held up toward thedriver for viewing and touch interface interaction according to anembodiment of the present invention.

FIG. 18 is another diagram of a driver and a mobile device held uptoward the driver for viewing and touch interface interaction accordingto an embodiment of the present invention.

In the appended figures, similar components and/or features can have thesame reference label. Further, various components of the same type canbe distinguished by following the reference label by a dash and a secondlabel that distinguishes among the similar components. If only the firstreference label is used in the specification, the description isapplicable to any one of the similar components having the same firstreference label irrespective of the second reference label.

DETAILED DESCRIPTION

Embodiments of the present invention utilize mobile devices to provideinformation on a user's behaviors during transportation. For example, amobile device carried by a user could be used to analyze drivingbehavior. Important driving behaviors that can be identified by someembodiments are device usage behavior and device interaction or touchbehavior, which is indicative of distraction or inattentiveness to oneor more driving tasks. The driving tasks may include speed, speed limit,direction, school zones, other vehicles, and/or other critical drivingtasks.

As discussed further herein, some embodiments described use approachesto collecting and analyzing driving data similar to the approachesdescribed in U.S. patent application Ser. No. 15/149,603, filed May 9,2016, entitled “METHODS AND SYSTEMS FOR SENSOR-BASED VEHICLEACCELERATION DETERMINATION” (“the '603 application”), U.S. patentapplication Ser. No. 15/149,613, filed May 9, 2016, entitled “METHODSAND SYSTEMS FOR SENSOR-BASED DRIVING DATA COLLECTION” (“the '613application”), U.S. patent application Ser. No. 14/749,232, filed Jun.24, 2015, entitled “METHODS AND SYSTEMS FOR ALIGNING A MOBILE DEVICE TOA VEHICLE” (“the '232 application”), and U.S. patent application Ser.No. 15/249,967, filed Aug. 29, 2016, entitled “METHODS AND SYSTEMS FORPRESENTING COLLECTED DRIVING DATA” (“the '967 application”), all ofwhich are incorporated by reference herein in their entireties for allpurposes (collectively “the Incorporated Applications”). Specificexamples of the use of different embodiments disclosed in theIncorporated Applications are provided herein, although other approachesdescribed in the Incorporated Applications may be implemented inconjunction with some embodiments of the present invention as well.

FIG. 1 is a system diagram illustrating a system 100 for collectingdriving data according to an embodiment of the present invention. System100 includes a mobile device 101 having a number of differentcomponents. Mobile device 101 includes a sensor data block 105, a dataprocessing block 120, a data transmission block 130, and a notificationblock 140. The sensor data block 105 includes data collection sensors aswell as data collected from these sensors that are available to mobiledevice 101. This can include external devices connected via Bluetooth,USB cable, etc. The data processing block 120 includes storage 126, andmanipulations done to the data obtained from the sensor data block 105by processor 122. This includes, but is not limited to, analyzing,classifying, characterizing, subsampling, filtering, reformatting, etc.Data transmission block 130 includes any transmission of the data offthe phone to an external computing device that can also store andmanipulate the data obtained from sensor data block 105. The externalcomputing device can be, for example, a server 150. Server 150 cancomprise its own processor 152 and storage 156. In one embodiment,notification block 140 reports the results of analysis of sensor dataperformed by the data processing block 120 to a user of the mobiledevice 101 via a display (not shown). The functions of notificationblock 140 are described further in U.S. Provisional Patent ApplicationNo. 62/286,218, filed Jan. 22, 2016, which is herein incorporated byreference in its entirety.

Some embodiments of the present invention are described using exampleswhere driving data is collected using mobile devices 101, and theseexamples are not limited to any particular mobile device. As examples, avariety of mobile devices including sensors such as accelerometers 112,gyroscopes 116, magnetometers 114, microphones 118, compasses 119,barometers 113, location determination systems such as globalpositioning system (GPS) receivers 110, communications capabilities, andthe like are included within the scope of the invention. Exemplarymobile devices include smart watches, fitness monitors, Bluetoothheadsets, tablets, laptop computers, smart phones, music players,movement analysis devices, and other suitable devices. One of ordinaryskill in the art, given the description herein, would recognize manyvariations, modifications, and alternatives for the implementation ofembodiments.

To collect data associated with the driving behavior of a driver, one ormore sensors on mobile device 101 (e.g., the sensors of sensor datablock 105) are operated close in time to a period when mobile device 101is with a driver when operating a vehicle—also termed herein “a drive”or “a trip”. With many mobile devices 101, the sensors used to collectdata are components of the mobile device 101, and use power resourcesavailable to mobile device 101 components, e.g., mobile device batterypower and/or a power source external to mobile device 101.

Some embodiments use settings of a mobile device to enable differentfunctions described herein. For example, in Apple IOS, and/or AndroidOS, having certain settings enabled can enable certain functions ofembodiments. For some embodiments, having location services enabledallows the collection of location information from the mobile device(e.g., collected by global positioning system (GPS) sensors, andenabling background app refresh allows some embodiments to execute inthe background, collecting and analyzing driving data even when theapplication is not executing. For some embodiments, the operating systemof mobile device 101 can enable reporting of the screen state of mobiledevice 101 (i.e., whether the screen is on or off at a particular pointin time).

FIG. 2 shows a system 200 for collecting driving data that can include aserver 201 that communicates with mobile device 101. In someembodiments, server 201 provides functionality using componentsincluding, but not limited to vector analyzer 258, vector determiner259, external information receiver 212, classifier 214, data collectionfrequency engine 252, driver detection engine 254, scoring engine 290,and device usage engine 256. These components are executed by processors(not shown) in conjunction with memory (not shown). Server 201 alsoincludes data storage 256. It is important to note that, while notshown, one or more of the components shown operating within server 201can operate fully or partially within mobile device 101.

To collect data associated with the driving behavior of a driver, one ormore sensors on mobile device 101 (e.g., the sensors of sensor datablock 105) are operated close in time to a period when mobile device 101is with the driver when operating a vehicle—also termed herein “a drive”or “a trip”. Once the mobile device sensors have collected data (and/orin real time), some embodiments analyze the data to determineacceleration vectors for the vehicle, as well as different features ofthe drive. Examples of processes to detect and classify driving featuresusing classifier 214, and determine acceleration vectors using vectoranalyzer 258 and vector determiner 259. In one embodiment, external data(e.g., weather) can be retrieved and correlated with collected drivingdata.

As discussed herein, some embodiments can transform collected sensordata (e.g., driving data collected using sensor data block 105) intodifferent results, including, but not limited to, estimates of theoccurrence of times where a driver was using the device. Such functionsmay be performed by device usage engine 256, for example. Examples ofcollecting driving data using sensors of a mobile device are describedherein. Examples of analyzing collected driving data to detect theoccurrence of device usage are also described herein. Notifications ofdriving events can be made via notification block 140 of mobile device101 in one embodiment.

As discussed further in U.S. Provisional Patent Application No.62/286,218 and U.S. Provisional Patent Application No. 62/346,013,herein incorporated by reference in their entireties, some embodimentsanalyze collected driving data and assign scores based on differentcriteria. Some embodiments use scoring engine 290 to analyze relevantdata and rules, and generate scores for embodiments.

Although shown and described as being contained within server 201, it iscontemplated that any or all of the components of server 201 may insteadbe implemented within mobile device 101. It is further contemplated thatany or all of the functionalities described herein may be performedduring a drive, in real time, or after a drive.

FIG. 3 is a flowchart 300 of the capture, analysis and use ofattentiveness data associated with a driver according to an embodimentof the present invention. The method described in FIG. 3 can use, forexample, approaches described in the Incorporated Applications to detectand measure the movement of a mobile device relative to a movingvehicle.

As described within the Incorporated Applications, this detection,measurement, and analysis of the movement of a mobile device can beaccomplished by collecting and storing sensor data from the mobiledevice (e.g., from sensors in sensor data block 105), then analyzing thedata at server 201 (e.g., using vector analyzer 258, vector determiner259, and/or classifier 214). In some embodiments, this analysis canadditionally or alternatively be performed using resources in the mobiledevice (e.g., data processing block 120 in mobile device 101).

At processing block 310, measurements are obtained indicating that themobile device is moving relative to the vehicle. In different examples,a measured movement can be minor (e.g., a mobile device sliding sidewayswithin a cup holder), or more substantial (e.g., the mobile device beingpicked up out of the cup holder and held to the ear of a driver).

At decision block 320, the movement measurements are analyzed todetermine whether they are indicative of a particular type of event(e.g., driver inattentiveness) occurring with respect to mobile device101 in a vehicle. In some embodiments, this particular type of event isassociated with use by the driver of mobile device 101 such that thedriver of the vehicle is potentially not paying attention to drivingtasks (e.g., the driver is distracted from driving tasks by mobiledevice 101). For convenience, as used herein, inattentiveness,distraction, failing to pay attention, and/or other similar terms andphrases broadly signify a driver not paying proper attention to tasksassociated with operating the vehicle.

It is important to note that, for any analysis described herein,exclusions, thresholds and other limitations can be applied to allow fortuning of results. For example, certain types of applications (e.g.,navigation applications, hands free phone calls) can be excluded in someembodiments from being assessed as distracting. In addition, a thresholdcan be applied to different analysis (e.g., a certain amount ofmovements must be detected to conclude a mobile device is being used).

As discussed below, not all movement measurements are associated withthe driver of the vehicle, and not all uses of mobile device by thedriver 101 lead to inattentiveness. It should be appreciated thatdifferent embodiments described herein provide the framework formeasuring inattentiveness, but the thresholds and classifications usedfor this analysis are tunable. The definitions of distraction used byembodiments can be changed at any time, if needed (e.g., for someapplications, any use of a mobile device is distracting to some degree,while for other applications, hands free use of a mobile device with aBluetooth headset is not distracting). Some entities may want toflexibly define the potential for distraction of a given activity basedon the demographics of the driver (e.g., hands-free use of a mobiledevice by a 21 year old person is distracting to some degree, whilehands-free use by a 45 year old person is not). This incorporation of awide variety of relevant factors in determining distraction is describedfurther herein, and different factors can be applied at different phasesin the analysis (e.g., picking up a mobile device can be determined tobe indicative of driver inattentiveness at decision block 320, but lateranalysis determines that the mobile device is being used by a passengerand is not distracting to the driver).

Examples of which types of movement measurements may be found, by someembodiments, indicative of distraction by decision block 320, and howthese movement measurements may be collected and analyzed, is discussedherein.

At processing block 330, the activity being performed with mobile device101 is estimated by some embodiments. Examples of estimated activitiesinclude, but are not limited to: (1) mobile device 101 is estimated tobe being held to the driver's (or a passenger's) ear, (2) mobile device101 is estimated to be being held by the driver (or passenger) such thatthe screen can be viewed, (3) mobile device 101 is being held forviewing and the driver (or passenger) is interacting with the device(e.g., purposefully touching the touch screen and/or hardware keyboardof the mobile device), (4) mobile device 101 is being moved from onelocation in the vehicle (e.g., a pocket, the cup holder, etc.) toanother location (e.g., a bracket mount closer to the driver's field ofview). Discussion of the types of processes used by some embodiments todifferentiate between the driver of a vehicle and passengers arediscussed herein, including with the descriptions of FIGS. 12 and 13.

In some embodiments, the estimate of the activity being performed withmobile device 101 at processing block 330 is accomplished by theinterpretation of movement measurements alone (e.g., one embodiment isnot receiving status information from mobile device 101 as to whatapplications (e.g., texting, email, phone, etc.) are being executed onthe device). It is contemplated that mobile devices 101 may vary intheir ability to report this status information in some embodiments. Forexample, an application executing on some iPhones, using some versionsof Apple IOS, may not be able to receive the status of other executingapplications, for any reason. With other mobile devices (e.g., thoseoperating with Android OS), an application can determine which otherapplications are executing.

Additional examples of types of activities which can be estimated byprocessing block 330, how these estimates are made, and the potentialsignificance of these estimates, vis-a-vis a determination ofdistraction, are discussed herein.

At processing block 350, in some embodiments, based on the time frame ofthe potentially distracting activity, aggravating and/or mitigatingfactors are identified. Though the attached figures illustratingembodiments should be interpreted such that, in some embodiments, allsteps/components could be optional, combined with other steps/componentsand/or performed by different steps/components, processing block 350 hasa dotted line to further emphasize the optional nature of this stage.This stage should not be interpreted to be more or less optional thanother figure elements that do not have dotted lines.

Aggravating and/or mitigating factors may be identified bycross-referencing one or more of pieces of external information thatinclude, but are not limited to: the time period of the identifiedactivity (e.g., from 8:12 PM to 8:22 PM), the location of the identifiedactivity from GPS measurements (e.g., Maple Road), the demographics ofthe person identified to be the driver of the vehicle (e.g., 22 year oldmale), the reported weather conditions at the location (e.g., snowing),any special status of the location (e.g., a school zone), the speed ofthe vehicle (e.g., 15 miles per hour), time of day (e.g., at dusk),calendar date (e.g., Christmas Eve), and/or any other similar types ofinformation relevant to how distracting an activity could be. Examplesof potential mitigating factors include, but are not limited to: theslow speed of the vehicle at the time of the event, the experience ofthe driver, the lack of traffic on the road, and the like.

An additional discussion of this type of cross-referencing with externaldata sources is found in the Incorporated Applications (e.g., gradinghow bad a hard braking event was by cross-referencing the event with theweather, time of day, school zone, etc.). It is contemplated that abroad variety of data sources can be combined to provide additionalaggravating and mitigating factor relevant to distracted driving.

At processing block 360, based on the analysis of blocks 310-350, anestimate of the level of driver inattentiveness during a particular timeperiod may be estimated. In some embodiments, this estimate is madeusing approaches described in the Incorporated Applications with respectto other driving behaviors. For example, by the processes described inthe '967 application, braking events, speeding, mobile device use andother detected behaviors are analyzed and a score is generated.

Once an estimate is completed by some embodiments, a score (e.g., ametric, grade and/or any other indication likely to understood bydrivers) can be generated. For example, letter grades may provide aneasy-to-understand rating for different levels of distraction (e.g., an“F” assigned to a single event, and/or a drive that contained the event,where an incident was detected in which a driver sent a text messagewhile the vehicle was moving). As described herein, and in theIncorporated Applications, grades assigned to smaller behavior features(e.g., a single event), can be weighted and/or aggregated into anevaluation of larger behavior features (e.g., a whole drive, a month ofdriving, a user's entire driving history, the driver's current behaviorrating, and/or other similar features).

FIG. 4A is a flowchart 400A illustrating a mobile device usage detectionmethod according to an embodiment of the invention. The method may beperformed by mobile device 101 and/or server 150 of FIG. 1, or mobiledevice 101 and/or server 201 of FIG. 2. At processing block 405,movements of a device are received. The device may be, for example,mobile device 101 of FIGS. 1 and/or 2. The movements may be measured byone or more sensors within sensor data block 105 of FIG. 1, such as theaccelerometer, the gyroscope, the GPS and/or the magnetometer. Themeasurements may include the non-gravitational acceleration of thedevice in the x, y, and z directions; the gravitational acceleration ofthe phone in the x, y, and z directions; the yaw, roll, and pitch of thedevice; the derivatives of these measurements; the gravity differenceangle of the device; and the difference in normed gravitationalacceleration of the device. In one embodiment, the movements of thedevice may be measured in intervals, e.g., over a period of 10 seconds.

At processing block 410, the movements of the device are classified asdevice usage movements based on characteristics of the device movements.Device usage movements as used herein refer to movements of the devicethat are indicative of usage of the device by the user. Exemplarycharacteristics include changes in angle of the device with respect togravity. Alternatively or additionally, the device movements may beclassified as device usage movements based on a comparison of the devicemovements to stored movements that are known or confirmed to be deviceusage movements. Alternatively or additionally, the device movements maybe classified as device usage movements if the number of devicemovements is above a threshold number. For example, if the device moves10 times in 30 seconds, the movements may be classified as device usagemovements, because the probability of the device moving that many timesin that time period without usage is low.

At processing block 415, one or more filters are applied to the deviceusage movements. For example, the direction of the user interface of thedevice can be determined, and device usage movements occurring while thedevice is facing away from the user can be filtered out. The directionof the user interface of the device can be determined, for example, bythe magnetometer, or by negative z-gravity-acceleration. In anotherexample, device usage movements occurring while the vehicle is notmoving (or moving at a speed below a threshold) can be filtered out.Whether or not the vehicle is moving can be determined, for example,from measurements made by the sensors within the device.

In still another example, device usage movements occurring while thedevice is face down (i.e., in a downward orientation) can be filteredout. For example, the device movements can be scanned over an interval(e.g., every 1/10 second) for a large gravity difference angle spike,which is indicative of the device being picked up. Device usagemovements can be filtered out, for example, if they occur prior to thedevice being picked up.

At processing block 420, the filtered device usage movements can beoutputted for further processing, display, storage, etc. For example,the filtered device usage movements can be used to compute summarystatistics, such as max, min, mean, med-abs-dev, covariance,percentiles, range, skew, kurtosis, correlation coefficients, zerocrossings, entropy, and/or power. The summary statistics may then beused to classify future movements of the device. Further, the summarystatistics may be used as input features to train a machine learningensemble. In another example, the filtered device usage movements can beadded to the stored device usage movements to help classify futuredevice usage movements.

In still another example, an alert can be generated to the userregarding the filtered device usage movements. For example, an audioalert or textual notification can be made to the user regarding thelength of time the user was using the device during the trip, asdiscussed further in U.S. Provisional Patent Application No. 62/286,218,herein incorporated by reference in its entirety.

At optional processing block 425, the filtered device usage movementscan be classified as device touch movements based on characteristics ofthe filtered device usage movements. Device touch movements as usedherein are movements of the device indicative of touching the device bythe user. For example, when a user interface of the device is repeatedlytapped by a driver, it can cause a frequency vibration in some degreethat may be detectable by movements sensors, such as an accelerometer orgyroscope. In one example, an alert can be generated to the userregarding the device touch movements, such as is discussed further inU.S. Provisional Patent Application No. 62/286,218, herein incorporatedby reference in its entirety.

FIG. 4B is a flowchart 400B illustrating a driver identification methodusing mobile device usage movements according to an embodiment of theinvention. The steps of flowchart 400B may be performed at any point offlowchart 400A after the device usage movements are classified atprocessing block 410. The method may be performed by mobile device 101and/or server 150 of FIG. 1, or mobile device 101 and/or server 201 ofFIG. 2. At processing block 430, movements and stops of the vehicle aredetermined using the movements of the device as obtained from one ormore sensors, for example.

At decision block 435, it is determined whether the device usagemovements begin during a deceleration or stop in the movements of avehicle. If not, then the movements and stops of the vehicle continue tobe monitored at processing block 430. If so, then the process continuesat decision block 440. At decision block 440, it is determined whetherthe device usage movements end during a stop or an acceleration in themovements of the vehicle. If not, then the movements and stops of thevehicle continue to be monitored at processing block 430. If so, thenthe user is classified as a driver of the vehicle at processing block445.

The classification of the user as a driver of a vehicle can be used inany number of ways. For example, the trip by the driver can be assigneda score based at least in part on the filtered device usage movements.Scoring of trips is discussed further in U.S. Provisional PatentApplication No. 62/286,218, herein incorporated by reference in itsentirety. Further disclosure with respect to driver identification canbe found in U.S. patent application Ser. No. 14/139,510, filed Dec. 23,2013, entitled “METHODS AND SYSTEMS FOR DRIVER IDENTIFICATION”, hereinincorporated by reference in its entirety. In an additional oralternative embodiment, the classification of the user as a driver of avehicle according to this method can be combined with driverclassifications according to other methods to determine an overallprobability that the user is the driver.

FIG. 4C is a flowchart 400C illustrating a driver identification methodusing gravitational acceleration movements according to an embodiment ofthe invention. The steps of flowchart 400C may be performed at any pointof flowchart 400A after the device usage movements are classified atprocessing block 410. The method may be performed by mobile device 101and/or server 150 of FIG. 1, or mobile device 101 and/or server 201 ofFIG. 2.

At processing block 450, a plurality of gravitational accelerationmovements of the device are determined from the device usage movements.At processing block 455, the plurality of gravitational accelerationmovements are corresponded to one or more orientations of the device.The orientations of the device may include, for example, landscapetilted up, landscape flat, tilted left portrait, and the like.

At processing block 460, the one or more orientations of the device areclassified as driver orientations or passenger orientations. Forexample, an orientation of the device may be classified as a driverorientation if that orientation is more consistent with device usage bya driver than by a passenger. In one embodiment, the driver orientationsmay include a tilted left portrait orientation. An orientation of thedevice may be classified as a passenger orientation if that orientationis more consistent with device usage by a passenger than by a driver. Inone embodiment, the passenger orientations may include landscapeorientations.

At processing block 465, a duration of time spent by the device in thedriver orientations and the passenger orientations is determined. Forexample, a total amount of time spent by the device in varying landscapeorientations can be determined as the amount of time spent by the devicein the passenger orientations. Similarly, a total amount of time spentby the device in a tilted left portrait orientation can be determined asthe amount of time spent by the device in the driver orientations.

At decision block 470, it is determined whether the duration of timespent by the device in the driver orientations is greater than theduration of time spent by the device in the passenger orientations. Thisdetermination can be made, for example, by comparing the duration oftime spent by the device in the driver orientations to the duration oftime spent by the device in the passenger orientations. If yes, themethod continues at processing block 480, and the user is classified asa driver of the vehicle. If no, the method continues at processing block475, and the user is classified as a passenger of the vehicle.

The classification of the user as a driver of a vehicle can be used inany number of ways. For example, the trip by the driver can be assigneda score based at least in part on the filtered device usage movements.Scoring of trips is discussed further in U.S. Provisional PatentApplication No. 62/286,218, herein incorporated by reference. Furtherdisclosure with respect to driver identification can be found in U.S.patent application Ser. No. 14/139,510, filed Dec. 23, 2013, entitled“METHODS AND SYSTEMS FOR DRIVER IDENTIFICATION”, herein incorporated byreference in its entirety. In an additional or alternative embodiment,the classification of the user as a driver of a vehicle according tothis method can be combined with driver classifications according toother methods to determine an overall probability that the user is thedriver.

Once the trip is completed, collective data may be collected and used.The data may include the number of times that device usage begins asspeed decreases to a stop; the number of times that device usage ends asspeed increases from a stop; the total amount of time the device is usedduring the trip; the lengths of the device usage periods; and the amountof time that the device spends in a variety of positions while the useris/is not using the device, and while the user is/is not moving quickly.The positions may include landscape mode, diagonal, upright, flat, leftfacing, right facing, upside down, etc. In one example, summarystatistics can be calculated based on the classification of the user asa driver and this collected data. For example, the statistics can beused as input features to a classification algorithm that determineswhether the user is a driver or a passenger in future trips.

FIG. 5 is a flowchart 500 illustrating a mobile device usage detectionmethod according to an embodiment of the invention. The method may beperformed by mobile device 101 and/or server 150 of FIG. 1, and/ormobile device 101 and/or server 201 of FIG. 2. Screen state data 505 andsensor data 510 are collected over one or more time intervals frommobile device 101 during a trip in a vehicle. The time intervals may ofany length, such as, for example, 10 second time intervals. Screen statedata 505 may be determined by an operating system of mobile device 101,and may indicate whether the screen of mobile device 101 is on or off ata given point in time. Sensor data 510 may include any data collected bysensor data block 105 of mobile device 101, such as, for example,gyroscope data, accelerometer data, global positioning system (GPS)data, and the like.

At processing block 515, a time series of usage probabilities aredetermined over the time intervals based on a comparison of the screenstate data 505 and sensor data 510 to known usage data. In someembodiments, a machine learning classifier may be applied to the screenstate data 505 and sensor data 510. For example, the screen state data505 and sensor data 510 can be used to compute summary statistics, suchas max, min, mean, med-abs-dev, covariance, percentiles, range, skew,kurtosis, correlation coefficients, zero crossings, entropy, and/orpower. The summary statistics may then be used to classify futuremovements of the device. Further, the summary statistics may be used asinput features to train a machine learning ensemble. In another example,the screen state data 505 and sensor data 510 can be added to the storeddevice usage movements to help classify future device usage movements.

The usage probabilities comprise probabilities that the user of mobiledevice 101 is interacting with mobile device 101 during the trip overeach time interval. The known usage data may comprise previouslycollected screen state date and/or sensor data from previous trips takenby the user of mobile device 101 and/or other users of other mobiledevices, with the data labeled according to known usage periods. Theusage periods may be known, for example, because they were explicitlyreported as usage periods by the users of the mobile devices. In anotherexample, the usage periods may have been determined to be known usageperiods according to the embodiments described herein or according toany other device usage detection method. For example, the usage periodsmay have been determined to be known usage periods according to thesystems and methods described in U.S. Provisional Patent Application No.62/318,983, filed Apr. 6, 2016, and/or U.S. Provisional PatentApplication No. 62/219,989, filed Sep. 17, 2015, herein incorporated byreference in their entireties.

At processing block 520, the sensor data 510 is validated for each timeinterval. In one embodiment, sensor data 510 is validated for a timeinterval if there is no significant sensor dropout during the timeinterval. In an additional or alternative embodiment, sensor data 510 isvalidated for a time interval if the user of mobile device 101 isdriving faster than a threshold speed (e.g., 2.6 m/s) during the timeinterval. This prevents data within sensor data 510 from being validatedif the user is using mobile device 101 while stopped, slowing to a stop,driving at a slow rate of speed, etc. In an additional or alternativeembodiment, sensor data 510 is validated if the screen state is “on”during the time interval, as indicated by the screen state data 505. Anyof a number of other criteria may be used to determine whether thesensor data 510 should be validated for a time interval. For example,sensor data 510 may be validated only if mobile device 101 has anegative grav-z acceleration. In another example, sensor data 510 may bevalidated only if sensor data 510 is not within a threshold time periodof the end of the trip.

At decision block 525, it is determined whether mobile device 101 is ina mount in the vehicle during each time interval of the trip based on acomparison of sensor data 510 to known mount data. This determinationmay be made, for example, by a mount detection engine within deviceusage engine 256, which may be implemented in mobile device 101, server150, and/or server 201. In one example, a machine learning classifiermay be applied to the sensor data 510. The known mount data may comprisepreviously collected sensor data from previous trips taken by the userof mobile device 101 and/or other users of other mobile devices, withthe data labeled according to known mount periods. The mount periods maybe known, for example, because they were explicitly reported as periodsthat the mobile devices were mounted in the vehicle by the users of themobile devices. In another example, the mount periods may have beendetermined to be known mount periods according to any other device mountdetection method.

For example, the sensor data 510 can be used to compute summarystatistics, such as max, min, mean, med-abs-dev, covariance,percentiles, range, skew, kurtosis, correlation coefficients, zerocrossings, entropy, and/or power. The summary statistics may then beused to classify future movements of the device as mount periods.Further, the summary statistics may be used as input features to train amachine learning ensemble. In another example, the sensor data 510 canbe added to the known mount data to help classify future mountmovements.

If it is determined that mobile device 101 is in a mount at decisionblock 525, the method continues at decision block 530. At decision block530, it is determined whether a GPS application is active on mobiledevice 101 during each time interval during which mobile device 101 wasin a mount. This determination may be made, for example, by a GPS activedetection engine within device usage engine 256, which may beimplemented in mobile device 101, server 150, and/or server 201. A GPSactive detection engine is described in further detail herein withrespect to FIG. 6.

In one example, a machine learning classifier may be applied to variousdata sources described in FIG. 6 to compare sensor data 510 and/or anyother collected data from mobile device 101 to known GPS active data.The known GPS active data may comprise previously collected data fromprevious trips taken by the user of mobile device 101 and/or other usersof other mobile devices, with the data labeled according to known GPSactive periods. The GPS active periods may be known, for example,because they were explicitly reported as periods that a GPS applicationwas being used on the mobile device, as reported by the user of themobile device. In another example, the GPS active periods may have beendetermined to be known GPS active periods according to any other GPSactive detection method. In another embodiment, it may be determineddirectly from the operating system of mobile device 101 that a GPSapplication is in the foreground of mobile device 101.

With respect to the machine learning classifier, the sensor data 510 canbe used to compute summary statistics, such as max, min, mean,med-abs-dev, covariance, percentiles, range, skew, kurtosis, correlationcoefficients, zero crossings, entropy, and/or power. The summarystatistics may then be used to classify future movements of the deviceas GPS active periods. Further, the summary statistics may be used asinput features to train a machine learning ensemble. In another example,the sensor data 510 can be added to the known GPS active data to helpclassify future GPS active movements.

If it is determined that mobile device 101 has an active GPS applicationat decision block 530, the method proceeds at decision block 535. Atdecision block 535, it is determined whether the usage probability forthe time intervals in which the GPS is active on mobile device 101 isgreater than a first threshold. The first threshold may be relativelyhigh so as not to identify distraction when mobile device 101 isproperly being used as a GPS device in a mount.

If it is determined that the usage probability for a time interval inwhich the GPS is active on mobile device 101 exceeds the firstthreshold, the method proceeds at processing block 540. At processingblock 540, the time interval is flagged as a device usage period if itwas determined to be valid at processing block 520. If it is determinedthat the usage probability for a time interval in which the GPS isactive on mobile device 101 does not exceed the first threshold, themethod proceeds at processing block 545. At processing block 545, thetime interval is discarded.

If it is determined that mobile device 101 does not have an active GPSapplication (i.e., mobile device 101 has a GPS application that isinactive, mobile device 101 does not have a GPS application, etc.) atdecision block 530, the method proceeds at decision block 550. Atdecision block 550, it is determined whether the usage probability forthe time intervals in which there is no active GPS application on mobiledevice 101 is greater than a second threshold. The second threshold maybe relatively low so as to identify distraction when mobile device 101is being used for purposes other than GPS purposes while it is mounted.In one embodiment, the second threshold is lower than the firstthreshold.

If it is determined that the usage probability for a time interval inwhich there is no active GPS application on mobile device 101 exceedsthe second threshold, the method proceeds at processing block 555. Atprocessing block 555, the time interval is flagged as a device usageperiod if it was determined to be valid at processing block 520. If itis determined that the usage probability for a time interval in whichthere is no active GPS application on mobile device 101 does not exceedthe second threshold, the method proceeds at processing block 560. Atprocessing block 560, the time interval is discarded.

If it is determined that mobile device 101 is not in a mount at decisionblock 525, the method continues at decision block 565. At decision block565, it is determined whether mobile device 101 is in a downward orupward orientation. This determination may be made, for example, byanalyzing sensor data 510. A “downward orientation” of mobile device 101may considered any orientation in which the screen of mobile device 101is not facing or unviewable the user of mobile device 101. An “upwardorientation” of mobile device 101 may be considered any orientation inwhich the screen of mobile device 101 is facing or viewable to the userof mobile device 101.

If it is determined that mobile device 101 is in a downward orientationat decision block 565, the method continues at decision block 570. Atdecision block 570, it is determined whether the usage probability for atime interval in which mobile device 101 is in a downward orientationexceeds a third threshold. The third threshold may be relatively highbecause it is unlikely that the user is using and/or distracted bymobile device 101 when the screen of mobile device 101 is pointing awayfrom the user.

If it is determined that the usage probability for the time interval inwhich mobile device 101 is in a downward orientation exceeds the thirdthreshold, the method proceeds at processing block 575. At processingblock 575, the time interval is flagged as a device usage period if itwas determined to be valid at processing block 520. If it is determinedthat the usage probability for the time interval in which mobile device101 is in a downward orientation does not exceed the third threshold,the method proceeds at processing block 380. At processing block 380,the time interval is discarded.

If it is determined that mobile device 101 is in an upward orientationat decision block 565, the method continues at decision block 585. Atdecision block 585, it is determined whether the usage probability for atime interval in which mobile device 101 is in an upward orientationexceeds a fourth threshold. The fourth threshold may be relatively lowbecause it is more likely that the user is using and/or distracted bymobile device 101 when the screen of mobile device 101 is pointingtoward the user.

If it is determined that the usage probability for the time interval inwhich mobile device 101 is in a upward orientation exceeds the fourththreshold, the method proceeds at processing block 590. At processingblock 590, the time interval is flagged as a device usage period if itwas determined to be valid at processing block 520. If it is determinedthat the usage probability for the time interval in which mobile device101 is in an upward orientation does not exceed the fourth threshold,the method proceeds at processing block 595. At processing block 595,the time interval is discarded.

Once the method has been completed for all of the time intervals of thetrip, the time intervals flagged as device usage periods may beconsolidated into continuous usage periods if they are within athreshold time of each other. In one embodiment, device usage periodswithin a minute of each other may be consolidated. For example, ifdevice usage periods are identified between (A) minute 5 of a trip andminute 5:10, (B) minute 5:10 and minute 5:20, and (C) minute 5:40 andminute 5:50, then minute 5 to minute 5:50 may be consolidated into acontinuous usage period.

Once continuous usage periods are identified, they may be used toperform analytics or to take further actions. In one embodiment, thecontinuous usage periods may be calculated to assign a score to a trip.Scoring of trips is described further in U.S. Provisional PatentApplication No. 62/286,218 and U.S. Provisional Patent Application No.62/346,013, herein incorporated by reference in their entireties. Forexample, a percentage of the time intervals that are identified ascontinuous usage periods may be calculated. The percentage may then becompared to percentages from other users, such as an average percentage,to determine how the device usage behavior of the user of mobile device101 compares to other users. In one example, if the percentage for theuser of mobile device 101 is lower than an average percentage, a highscore may be assigned to the trip. If the percentage for the user ofmobile device 101 is higher than an average percentage, a low score maybe assigned to the trip.

Driver identification methods may also be applied in conjunction withthe method illustrated in FIG. 5. In one embodiment, the screen statedata 505 and sensor data 510 may be used to calculate summary statisticsfor time intervals in which the screen is on. A machine learningclassifier may be applied to the summary statistics to compare sensordata 510 to known driver usage data. The known driver usage data maycomprise previously collected sensor data from previous trips taken bythe user of mobile device 101 while driving and/or other drivers usingother mobile devices, with the data labeled according to known driverusage periods. The driver usage periods may be known, for example,because they were explicitly reported as periods that the mobile deviceswere being used by drivers in the vehicles. In another example, thedriver usage periods may have been determined to be known driver usageperiods according to any other driver usage detection method, such asthose disclosed in U.S. Provisional Patent Application No. 62/318,983,herein incorporated by reference in its entirety.

In some embodiments, the screen state data 505 and sensor data 510 canbe used to compute summary statistics, such as max, min, mean,med-abs-dev, covariance, percentiles, range, skew, kurtosis, correlationcoefficients, zero crossings, entropy, and/or power. The summarystatistics may then be used to classify future movements of the deviceas driver usage periods. Further, the summary statistics may be used asinput features to train a machine learning ensemble. In another example,the sensor data 510 can be added to the known driver usage data to helpclassify future driver usage movements.

In an additional or alternative embodiment, a user of mobile device 101may be classified as a driver of the vehicle if it is determined thatmobile device 101 is mounted at decision block 525. This classificationis made based on the assumption that drivers of vehicles are more likelyto mount their mobile devices in their vehicles than passengers.

FIG. 6 is a flowchart 600 illustrating the inputs into a GPS activedetection engine 635 according to an embodiment of the invention. TheGPS active detection engine 635 may be implemented in mobile device 101and/or server 150 of FIG. 1, and/or mobile device 101 and/or server 201of FIG. 2. Sensor data 610 is collected from mobile device 101 during atrip in a vehicle. Sensor data 610 may include any data collected bysensor data block 105 of mobile device 101, such as, for example,gyroscope data, accelerometer data, global positioning system (GPS)data, and the like. Sensor data 610 may also include any data collectedby any additional sensors or components of mobile device 101 that arenot illustrated in sensor data block 105. Sensor data 610 may be thesame as or different than sensor data 510 of FIG. 5.

Various types of data may be extracted from the sensor data 610, such asmount data 615, gravity vector data 620, acceleration data 625, and/orbattery level data 630. Mount data 615 may be a determination of whetheror not mobile device 101 is in a mount, as decided at decision block 525of FIG. 5, for example. Mount data 615 may be provided to GPS activedetector 635, and may be relevant because a GPS application may be morelikely to be active when mobile device 101 is mounted.

Gravity vector data 620 may comprise summary statistics of gravityvectors during the trip. Gravity vector data 620 may be provided to GPSactive detection engine 635. Gravity vector data 620 may be relevantbecause initiation and/or use of a GPS application may have certaingravity vectors associated with the movements of mobile device 101.

Acceleration data 625 may comprise summary statistics of horizontal andvertical acceleration during the trip. Acceleration data 625 may beprovided to GPS active detection engine 625. Acceleration data 625 maybe relevant because initiation and/or use of a GPS application may havecertain accelerations associated with the movements of mobile device101.

Battery level data 630 may comprise data provided by the operatingsystem of mobile device 101, for example, that allows calculation of aslope of the battery level curve. Battery level data may be provided toGPS active detection engine 625. Battery level data 630 may be relevantbecause use of a GPS application may drain the battery of mobile device101 faster than if a GPS application is not being used.

Mount data 615, gravity vector data 620, acceleration data 625, and/orbattery level data 630 may be input into GPS active detection engine 635for further analysis to determine whether a GPS application is active onmobile device 101. GPS active detection engine 635 may determine that aGPS application is active on mobile device 101, for example, by applyinga machine learning classifier to the data 615-630 to compare the data615-630 to known GPS active data, as described further herein. The knownGPS active data may comprise previously collected data from previoustrips taken by the user of mobile device 101 and/or other users of othermobile devices, with the data labeled according to known GPS activeperiods. The GPS active periods may be known, for example, because theywere explicitly reported as periods that a GPS application was beingused on the mobile device, as reported by the user of the mobile device.In another example, the GPS active periods may have been determined tobe known GPS active periods according to any other GPS active detectionmethod. In another embodiment, it may be determined directly from theoperating system of mobile device 101 that a GPS application is in theforeground of mobile device 101.

For example, the data 615-630 can be used to compute summary statistics,such as max, min, mean, med-abs-dev, covariance, percentiles, range,skew, kurtosis, correlation coefficients, zero crossings, entropy,and/or power. The summary statistics may then be used to classify futuremovements of the device as GPS active periods. Further, the summarystatistics may be used as input features to train a machine learningensemble. In another example, the data 615-630 can be added to the knownGPS active data to help classify future GPS active movements.

FIGS. 7A-B, 8A-B and 9A-B are graphs of actual mobile device usageintervals and accelerometer measurements during those intervals when auser is using a mobile device according to an embodiment of the presentinvention. For example, FIG. 7A shows time intervals in which a user wasplaying a game and taking a phone call on a mobile device. FIG. 7B showsthe accompanying x, y, and z acceleration data as collected by anaccelerometer of the mobile device before, during and after the usage.As shown in FIG. 7B, the x and z accelerations increased during theusage time periods, while the y acceleration decreased.

FIG. 8A shows time intervals in which a user was using differentapplications on a mobile device. FIG. 8B shows the accompanying x, y,and z acceleration data as collected by an accelerometer of the mobiledevice before, during and after the usage. As shown in FIG. 8B, the yand z accelerations increased during the usage time periods, while the xacceleration decreased.

FIG. 9A shows time intervals in which a user was using e-mail,applications and text messaging on a mobile device. FIG. 9B shows theaccompanying x, y, and z acceleration data as collected by anaccelerometer of the mobile device before, during and after the usage.As shown in FIG. 9B, the z acceleration increased, while the x and yaccelerations decreased, and/or the accelerations that were otherwisestable exhibited a large degree of variance.

The data collected and shown in FIGS. 7A-B, 8A-B and 9A-B may be used asknown usage data as described herein. For example, FIGS. 7B, 8B, and 9Bmay be labeled according to the known usage periods shown in FIGS. 7A,8A, and 9A. The usage periods may be known, for example, because theywere explicitly reported as usage periods by the users of the mobiledevices. The data in the known usage periods may then be used by amachine learning classifier to classify other usage periods showingsimilar acceleration characteristics. In another example, FIGS. 7B, 8Band 9B may be used by embodiments of the invention to predict mobiledevice usage behavior, and those predictions may be compared to FIGS.7A, 8A, and 9A to determine their accuracy.

For example, the data shown in FIGS. 7B, 8B and 9B can be used tocompute summary statistics, such as max, min, mean, med-abs-dev,covariance, percentiles, range, skew, kurtosis, correlationcoefficients, zero crossings, entropy, and/or power. The summarystatistics may then be used to classify future movements of the deviceas usage periods. Further, the summary statistics may be used as inputfeatures to train a machine learning ensemble. In another example, thedata shown in FIGS. 7B, 8B and 9B can be added to the known usage datato help classify future usage movements.

FIGS. 10A-10D are plots of sensor measurements and predicted usageintervals according to an embodiment of the invention. FIG. 10Aillustrates the speed of a vehicle 1010 over time as measured by mobiledevice 101 of FIGS. 1 and/or 2 during a trip, for example, as a line.The shaded areas of FIG. 10A represent predicted device usage periods1015, 1020. FIG. 10B illustrates the gravity diff angle 1025 over timeas measured by mobile device 101 of FIGS. 1 and/or 2 during a trip, forexample. FIG. 10C illustrates the screen on probabilities 1030 over timeof mobile device 101 of FIGS. 1 and/or 2 during a trip, for example. Theshaded area 1035 of FIG. 10C represents the screen of the mobile device101 in an “on” state. FIG. 10D illustrates the acceleration in the x1040, y 1045 and z 1050 directions over time as measured by mobiledevice 101 of FIGS. 1 and/or 2 during a trip, for example.

FIG. 11A is a plot of predicted device usage 1110 and actual usage 1120according to an embodiment of the invention. The plot of FIG. 11A may begenerated as output by device usage engine 256 of FIG. 2, for example,and/or by performing any or all of the methods of FIGS. 3-6. FIG. 11B isa plot of the actual or known device usage 1130 according to anembodiment of the invention. As shown in FIGS. 11A and 11B, thepredicted device usage periods provided by embodiments of the inventionaccurately predict actual device usage.

FIGS. 12-18 provide examples of different types of processes, used bysome embodiments, to collect and analyze movement measurements frommobile device 101. FIG. 12 is a diagram 1200 of a mobile device 101 in avehicle 1250, according to an embodiment. FIG. 12 depicts a vehicle 1250having a driver (not shown) where mobile device 101 is used to providemovement measurements that enable estimates of driving behavior.

In some embodiments, as described in the '603 application, using anextended Kalman filter applied to movement data from mobile devicesensors, a gravity vector (e.g., gravity vector 1240) for a mobiledevice (e.g., mobile device 101) in a moving vehicle (e.g., vehicle1250) moving in the direction indicated by reference numeral 1260.Generally speaking, in a moving vehicle, the determined gravity vectorof the mobile device as a function of time is always changing.

To determine a change in the orientation of mobile device 101 relativeto vehicle 1250, some embodiments analyze two gravity vectors associatedwith a time point (t), for example, gravity vectors G(t) before andafter the time point (e.g., at t−15 seconds and t+15 seconds). Bycomparing these gravity vectors over a time interval, a difference ingravity angle (e.g., measured in radians), and gravity magnitude can bedetermined 1240 (respectively termed herein, gravity angle differenceand gravity magnitude difference). It should be appreciated that alarger or smaller interval can also be used. It is also important tonote that, while embodiments described herein are described as operatingon stored data (i.e., after the drive has completed, not in real time),the components, principles and approaches described herein could alsooperate in substantially real-time (e.g., using a t−30 and t−15 intervalfor example to analyze gravity vectors, or other suitable approach).

In some embodiments, whether the mobile device remained stationarywithin the vehicle during a time interval can be estimated by comparingthe determined gravity angle difference and the determined gravitymagnitude difference to one or more thresholds. In an example of thisthreshold comparison, having a gravity angle difference above one (1)radian and a gravity magnitude difference above one (1) g indicate to anembodiment that the mobile device likely moved relative to the vehicleat time point (t). It is contemplated that different combinations ofthreshold values could be used, e.g., different gravity magnitudedifferences, different gravity angle differences, and/or requiringmultiple consecutive differences over thresholds before a change inorientation is estimated.

FIG. 13 is a diagram 1300 of a driver 1340 and a mobile device 101 in avehicle according to an embodiment. Mobile device 101 is in a cup holder1310. Vector 1260 shows the direction of the vehicle relative to driver1340. Gravity vector 1240 shows that mobile device 101 is angledbackwards in cup holder 1310. For some embodiments, this is the startingpoint of the drive, e.g., mobile device 101 is placed in cup holder1310, the drive begins, and any movement of mobile device 101 relativeto vehicle 1250 is detected and measured for distraction potential.Other example starting positions include, but are not limited to:mounted in a mobile device holder attached to a windshield, rear viewmirror, dashboard or other similar vehicle surface, placed on a seat, ina clothes pocket, and the like. It is important to note that, someembodiments do not need any indication of where mobile device 101 isplaced within a vehicle in order to perform the movement measurementanalysis described herein.

FIG. 14 is a diagram of a mobile device in a cup holder in a vehicleaccording to an embodiment. Differing from the placement of mobiledevice 101 in the cup holder of FIG. 13, angle 1450 shows that mobiledevice 101 is angled to the left (to the driver's right) in the cupholder 1310. It is important to note that, some embodiments to not needany particular orientation of mobile device 101 to be able to collectuseful movement measurements for embodiments.

Vector 1440 illustrates a direction perpendicular to the back (andfront) of mobile device 101. As discussed with the description of FIGS.17 and 18 below, this vector 1440 is used by some embodiments toestimate whether a person is interacting with the mobile device, e.g.,typing on a keyboard.

FIG. 15 is a diagram of a driver 1340, a mobile device 101-1, 101-2 andaccelerometer data according to an embodiment. When sitting in cupholder 1310, mobile device 101 is generally not moving relative to thevehicle. Even if mobile device 101 moves around within cup holder 1310,this movement can be subject to a threshold that would be exceededbefore triggering further analysis.

In contrast to a minor movement example (e.g., sliding around in cupholder 1310), FIG. 15 shows an example of a move of mobile device 101that could trigger analysis by some embodiments. In this example, driver1340 takes mobile device 101-1 in hand (hand not shown), lifts it up,toward him, and to the right. This movement can be detected and analyzedby accelerometer 112 and gyroscope 138. The measurements collected fromthis accelerometer 112, as mobile device 101 moves along track 1510, areshown in relation to X axis 1540, Y axis 1520 and Z axis 1530. In someembodiments, the movement shown in FIG. 15 is sufficient to triggerfurther analysis for distraction potential, e.g., mobile device 101 wasmounted, but now it is moving. Some embodiments assess that this couldbe because the driver is holding the device and typing on the screen, ormaking a phone call without a headset.

After mobile device 101 stops moving (e.g., it is moved from position101-1 to position 101-2), the change in orientation is measured. In thisexample, after the orientation measurement, the position 101-2 of mobiledevice 101 is depicted in FIG. 15. As shown in FIG. 15, mobile device101 is angled in such a way that it could be used as a telephone,without use of a hands free device. Based on the orientation 101-2 ofmobile device 101, in some embodiments, the estimated activity is thatdriver 1340 is holding mobile device 101 to her ear, and thus making aphone call without use of a headset.

It is worth noting the processes, used by some embodiments, todifferentiate between used of mobile device 101 by the driver of avehicle and non-drivers, e.g., passengers in either the front of thevehicle with the driver or in rearward seats. Using the example fromFIG. 15, mobile device 101 in position 101-2 is determined to bepointing at driver 1340, and thus some embodiments would estimate mobiledevice 101 was being used by the driver in that position. Similarly, ina different example, mobile device 101 can be facing the passenger'sseat, or rear seats (with accompanying accelerometer data suggesting amovement along the Z axis 1530 to the back seats). In this differentexample, driver 1340 would not be estimated by some embodiments to bedistracted by this movement.

FIG. 16 is a diagram of a driver 1340 and a mobile device 101 held uptoward the driver for screen viewing according to an embodiment. Asanother example of measuring movement and estimating activity, FIG. 16shows a view of driver 1340 from the back and mobile device 101. In thisexample, it can be seen that mobile device 101 also undergoes asignificant move (yet shorter than the move discussed with FIG. 15) fromcup holder 1310 to a position where screen 1620 is facing driver 1340.As with FIG. 15, in some embodiments, this move can trigger furtheranalysis of the position of mobile device 101. It is worth noting that,this analysis first of movement, then of position can, in someembodiments, improve the efficiency of the process (e.g., by reducingthe amount of movement measurements that are collected and stored forlater analysis or by reducing real-time analysis of movementmeasurements). It is also important to note that, different types ofmeasurement collection and analysis can be performed sequentially,simultaneously and/or combinations of the two approaches.

FIG. 17 is a diagram of a driver 1340 and a mobile device 101 held uptoward the driver for viewing and touch interface interaction accordingto an embodiment. This view (e.g., from the right side of driver 1340)shows another view of a mobile device 101 orientation where mobiledevice 101 is positioned in a way that, in some embodiments, is assessedto be indicative of a reading or typing action being performed by driver1340. It is contemplated that some embodiments can be configured todetect different orientations of mobile device 101 and analyze theseorientations to estimate other types of activities of driver 1340.

In some embodiments, movement and orientation of mobile device 101 thatis similar to the examples of FIGS. 16 and 17 signal a likelihood that,in addition to viewing screen 1620, driver 1340 may be interacting withinterface elements of mobile device 101 (termed herein as “interactions”or “touches”). It is contemplated that interactions (e.g., touching auser interface, typing on a user interface or hardware keyboard,swiping, tapping, and/or other input events) can be a distractingactivity for driver 1340. Some embodiments use different approaches todetermine whether different interactions are occurring.

FIG. 18 is another diagram of a driver 1340 and a mobile device 101 heldup toward the driver potentially for viewing and touch interfaceinteraction, according to an embodiment. Vector 1440, introduced abovein an example where mobile device 101 was still in cup holder 1310, isused by some embodiments to detect interactions with driver 1340. Forexample, when a keyboard (software or hardware) is repeatedly tapped bydriver 1340, it can cause a frequency vibration towards vector 1440 insome degree (e.g., detectable by movement sensors, such as accelerometer112 and gyroscope 116) (termed herein “device touch movements”).

In another example, detecting a single tap, then a tilting movement in aparticular direction can indicate a swiping gesture to some embodiments.It should be noted that, rather than using indications from theoperating system of mobile device 101 to determine specific applicationactivities (e.g., texting, social media posting, scrolling of content,etc.), some embodiments use device sensors to detect the underlying(potentially distracting) elements of these specific activities.

It is contemplated that the detection of different interactions ortouches can be measured, tuned and customized for individual users, orcan also be tuned for generic users. In some embodiments, it is acombination of the processes discussed herein (e.g., movement to anorientation, the orientation, and the detected device touch movements)that lead to the estimates of different activities.

Also, in some embodiments, individual processes described herein canlead to estimates. For example, if device usage movements are detectedin mobile device 101 while mobile device 101 is stationary in vehicle1250 (e.g., as with the example discussed with FIG. 13 where mobiledevice 101 is in cup holder 1310, or mounted on the vehicle dashboard),the device usage movements can be such that some embodiments estimatethat interaction is occurring with mobile device 101 by driver 1340(i.e., device touch movements).

It should be appreciated that the specific steps illustrated in FIGS.3-6 provide a particular method of collecting driving data using a modelaccording to an embodiment of the present invention. Other sequences ofsteps may also be performed according to alternative embodiments. Forexample, alternative embodiments of the present invention may performthe steps outlined above in a different order. Moreover, the individualsteps illustrated in FIGS. 3-6 may include multiple sub-steps that maybe performed in various sequences as appropriate to the individual step.Furthermore, additional steps may be added or removed depending on theparticular applications. One of ordinary skill in the art wouldrecognize many variations, modifications, and alternatives.

The examples and embodiments described herein are for illustrativepurposes only. It is contemplated that various modifications may be madein light of the embodiments described herein. These are to be includedwithin the spirit and purview of this application, and the scope of theappended claims, which follow.

What is claimed is:
 1. A system comprising: a mobile device comprising aplurality of sensors; a memory; and a processor coupled to the memory,wherein the processor is configured to perform operations including:obtaining a plurality of movement measurements over a plurality of timeintervals from at least one of the plurality of sensors in the mobiledevice of a user during a trip in a vehicle; classifying a subset of theplurality of movement measurements as mobile device usage movementsbased on characteristics of the subset, wherein the mobile device usagemovements are movements obtained from the at least one of the pluralityof sensors, and wherein the mobile device usage movements are indicativeof usage of the mobile device by the user based on the movements;filtering the mobile device usage movements; and outputting the filteredmobile device usage movements, wherein the classifying the subset of theplurality of movement measurements as the mobile device usage movementscomprises: determining a time series of usage probabilities over thetime intervals based on a comparison of the obtained plurality ofmovement measurements to known usage measurements, wherein the usageprobabilities include probabilities that the user is using the mobiledevice over the time intervals, and wherein the filtering the mobiledevice usage movements comprises: determining that the mobile device isnot in a mount in the vehicle during the trip by comparing the obtainedplurality of movement measurements to known mount measurements;determining if the mobile device is in a downward orientation or anupward orientation for the time intervals using the obtained pluralityof movement measurements; flagging the time intervals with the usageprobabilities greater than a first threshold if the mobile device is inthe downward orientation and the time intervals with the usageprobabilities greater than a second threshold if the mobile device is inthe upward orientation, wherein the first threshold is greater than thesecond threshold; and consolidating the flagged time intervals within athreshold time period into continuous periods of the usage.
 2. Thesystem of claim 1, wherein the usage of the mobile device by the userincludes interaction with the mobile device by the user.
 3. The systemof claim 1, wherein the classifying the subset of the plurality ofmovement measurements as the mobile device usage movements furthercomprises: extracting the subset of the plurality of movementmeasurements corresponding to the time intervals in which the usageprobabilities exceed one of the first and second thresholds.
 4. Thesystem of claim 1, wherein the filtering the mobile device usagemovements comprises: determining that the mobile device is in the mountin the vehicle during the trip by comparing the obtained plurality ofmovement measurements to known mount measurements; determining if aglobal positioning system (GPS) application is active on the mobiledevice by comparing the obtained plurality of movement measurements toknown GPS active data; flagging the time intervals with the usageprobabilities greater than the first threshold if the GPS application isactive and the time intervals with the usage probabilities greater thanthe second threshold if the GPS application is inactive or is notpresent on the mobile device, wherein the first threshold is greaterthan the second threshold; and consolidating the flagged time intervalswithin the threshold time period into continuous periods of the usage.5. The system of claim 4, further comprising: classifying the user ofthe mobile device as a driver of the vehicle on the trip afterdetermining that the mobile device is in the mount.
 6. The system ofclaim 1, wherein the filtering the mobile device usage movementscomprises: extracting the mobile device usage movements that occurredwhile a screen of the mobile device was on.
 7. The system of claim 1,further comprising: determining if the user of the mobile device is adriver of the vehicle on the trip by comparing the obtained plurality ofmovement measurements to known driver usage measurements.
 8. The systemof claim 1, further comprising: calculating a score for at least one ofthe user or the trip using the filtered mobile device usage movements,wherein the outputting the filtered mobile device usage movementsincludes outputting the score.
 9. The system of claim 1, wherein themobile device usage movements are movements of the mobile device sensedby the at least one of the plurality of sensors.
 10. The system of claim1, wherein the mobile device usage movements are movements of the mobiledevice caused by the user.
 11. A method comprising: obtaining aplurality of movement measurements over a plurality of time intervalsfrom at least one of a plurality of sensors in a mobile device of a userduring a trip in a vehicle; classifying a subset of the plurality ofmovement measurements as mobile device usage movements based oncharacteristics of the subset, wherein the mobile device usage movementsare movements obtained from the at least one of the plurality ofsensors, and wherein the mobile device usage movements are indicative ofusage of the mobile device by the user based on the movements; filteringthe mobile device usage movements; and outputting the filtered mobiledevice usage movements, wherein the classifying the subset of theplurality of movement measurements as the mobile device usage movementscomprises: determining a time series of usage probabilities over thetime intervals based on a comparison of the obtained plurality ofmovement measurements to known usage measurements, wherein the usageprobabilities include probabilities that the user is using the mobiledevice over the time intervals, and wherein the filtering the mobiledevice usage movements comprises: determining that the mobile device isnot in a mount in the vehicle during the trip by comparing the obtainedplurality of movement measurements to known mount measurements;determining if the mobile device is in a downward orientation or anupward orientation for the time intervals using the obtained pluralityof movement measurements; flagging the time intervals with the usageprobabilities greater than a first threshold if the mobile device is inthe downward orientation and the time intervals with usage probabilitiesgreater than a second threshold if the mobile device is in the upwardorientation, wherein the first threshold is greater than the secondthreshold; and consolidating the flagged time intervals within athreshold time period into continuous periods of the usage.
 12. Themethod of claim 11, wherein the usage of the mobile device by the userincludes interaction with the mobile device by the user.
 13. The methodof claim 11, wherein the classifying the subset of the plurality ofmovement measurements as the mobile device usage movements furthercomprises: extracting the subset of the plurality of movementmeasurements corresponding to the time intervals in which the usageprobabilities exceed one of the first and second thresholds.
 14. Themethod of claim 11, wherein the filtering the mobile device usagemovements comprises: determining that the mobile device is in the mountin the vehicle during the trip by comparing the obtained plurality ofmovement measurements to known mount measurements; determining if aglobal positioning system (GPS) application is active on the mobiledevice by comparing the obtained plurality of movement measurements toknown GPS active data; flagging the time intervals with the usageprobabilities greater than the first threshold if the GPS application isactive and the time intervals with the usage probabilities greater thanthe second threshold if the GPS application is inactive or is notpresent on the mobile device, wherein the first threshold is greaterthan the second threshold; and consolidating the flagged time intervalswithin the threshold time period into continuous periods of the usage.15. The method of claim 14, further comprising: classifying the user ofthe mobile device as a driver of the vehicle on the trip afterdetermining that the mobile device is in the mount.
 16. The method ofclaim 11, wherein the filtering the mobile device usage movementscomprises: extracting the mobile device usage movements that occurredwhile a screen of the mobile device was on.
 17. The method of claim 11,further comprising: determining if the user of the mobile device is adriver of the vehicle on the trip by comparing the obtained plurality ofmovement measurements to known driver usage measurements.
 18. The methodof claim 11, further comprising: calculating a score for at least one ofthe user or the trip using the filtered mobile device usage movements,wherein the outputting the filtered mobile device usage movementsincludes outputting the score.
 19. The method of claim 11, wherein themobile device usage movements are movements of the mobile device sensedby the at least one of the plurality of sensors.
 20. The method of claim11, wherein the mobile device usage movements are movements of themobile device caused by the user.